|
Thanks for your interest in LMFlow! A quick fix is to remove the following section in configs/ds_config_zero3.json:
"optimizer": {
"type": "AdamW",
"params": {
"lr": "auto",
"betas": "auto",
"eps": "auto",
"weight_decay": "auto"
}
},
DeepSpeed uses a nvcc/c++ version of adam to speed up the optimization, which requires a compilation process and may sometimes fail due to wrong version of nvcc/c++. To find the exact problem that causes this issue, could you please provide the full log of error message, so we could check that for you?
As for installation of torch, by following this section of README, torch should be automatically installed. Thanks! 😄
|